Feasible-side Global Convergence in Experimental Optimization

where u ∈ Ru are independent decision variables subject to the bounds u and u – the curly brackets (≺) denoting componentwise inequality – and φ, g : Ru → R are cost and constraint functions, respectively. The characteristic element of (1.1) is the presence of experimental functions, denoted by the subscript p (for “plant”), which may only be evaluated by conducting an experiment for a given choice of u and whose values cannot be known otherwise. In this work, the term “experiment” will be employed to denote a repeatable but expensive task, where “repeatable” means that carrying out the task once with the variables ua and again with ub will yield identical results if ua = ub, while “expensive” implies that carrying out the task either is financially costly (e.g., machining a very expensive space shuttle component), requires a lot of time (e.g., simulating one day of traffic behavior for a large metropolis), or may only be done very infrequently (e.g., producing a large batch of a pharmaceutical compound once every three months). Of course, such expenses are not mutually exclusive and may also occur together. By contrast, the constraints without the p subscript indicate numerical functions that can be easily evaluated for any given u without requiring any experiments. Consequently, we will refer to (1.1) as an experimental optimization problem. The first formal studies on methodologically solving such problems may be traced back to the 1940s, 50s, and 60s, with the works of Hotelling [46], Box [10, 9], Brooks [12, 13], and Spendley et al. [73] essentially representing the foundations of this field. The methods that came out of these works – namely, those of (experimental) steepest ascent, evolutionary operation, response-surface modeling, and the simplex algorithm – have remained popular to the present day and are still employed in a number of diverse applications [40, 44, 41, 42, 3, 4, 64]. Additionally, there are entire fields of research dedicated to solving problems that may be cast in the form of (1.1). We cite, as some examples that we have encountered: